In a single-chip color image sensor, image sensing elements for luminance information (e.g. green or white) are spatially separated by image sensing elements for chrominance information (e.g. red and blue or cyan and yellow), leaving gaps between the luminance samples in the luminance signal produced by the image sensor. These gaps are usually filled in by some signal processing method involving interpolation.
U.S. Pat. No. 4,176,373 issued Nov. 27, 1979 to P. L. P. Dillon, shows a method for processing sampled image signals produced by an image sensor having a checkerboard luminance sampling pattern. The method involves linear interpolation (equivalent to finding the mean of the neighboring samples). The patent discloses linear interpolation along a horizontal line, and mentions that the method could be carried out in two dimensions by using samples from neighboring lines.
FIG. 1 shows a color sampling pattern for a single-chip color image sensor. The luminance sampling elements, labeled G, are separated by chrominance sampling elements, labeled R and B. FIG. 2a represents an image of a sharp vertical line. The elements labeled 100 represent high brightness and the elements labeled 0 represent low brightness. FIG. 2b represents the sampled values obtained from sampling the image of FIG. 2a with the luminance sampling pattern of FIG. 1. When linear interpolation is applied to the values shown in FIG. 2b, using the values of the four nearest neighbors (i.e. top, bottom, right and left) surrounding each interpolation location, the pattern illustrated in FIG. 2c is obtained.
As seen in FIG. 2c, the originally sharp vertical edge has been modulated with a pattern corresponding to the luminance sampling pattern. Similarly, when a thin stripe, such as the vertical stripe shown in FIG. 3a is sampled (the resulting samples are shown in FIG. 3b), and reconstructed using linear interpolation between the four nearest neighbors, the stripe is modulated with a pattern corresponding to the luminance sampling pattern as shown in FIG. 3c.
In an image intended for human viewing, these reconstruction errors are most objectionable in geometrical details such as edges, lines and corners. In an image intended for machine interpretation, such as a robot vision system used in a factory, the reconstruction errors are most objectionable when they interfere with the efficient operation of the system. For example, when the reconstruction error results in ambiguity as to the identification of a part of an assembly line.
Of course, the reconstruction errors may be reduced by increasing the number of image sensing elements in the image sensor but this increases the cost and complexity of the image sensor.
There is a need therefore for a signal processing method and apparatus for providing interpolated values between the signal values in a sampled image signal, that reduces the reconstruction errors in an image reconstructed from the processed image signal.